Self-organized computation with unreliable, memristive nanodevices

Nanodevices have terrible properties for building Boolean logic systems: high defect rates, high variability, high death rates, drift, and (for the most part) only two terminals. Economical assembly requires that they be dynamical. We argue that strategies aimed at mitigating these limitations, such as defect avoidance/reconfiguration, or applying coding theory to circuit design, present severe scalability and reliability challenges. We instead propose to mitigate device shortcomings and exploit their dynamical character by building self-organizing, self-healing networks that implement massively parallel computations. The key idea is to exploit memristive nanodevice behavior to cheaply implement adaptive, recurrent networks, useful for complex pattern recognition problems. Pulse-based communication allows the designer to make trade-offs between power consumption and processing speed. Self-organization sidesteps the scalability issues of characterization, compilation and configuration. Network dynamics supplies a graceful response to device death. We present simulation results of such a network—a self-organized spatial filter array—that demonstrate its performance as a function of defects and device variation. (Some figures in this article are in colour only in the electronic version) 1. Nanoelectronics and computing paradigms Nanodevices are crummy1. High defect rates, high device variability, device ageing, and limitations on device complexity (e.g., two-terminal devices are much easier to build) are to be expected if we intend to mass produce nanoelectronic systems economically. Not only that, it is almost axiomatic among many researchers that such systems will be built from simple structures, such as crossbars, composed of nanodevices that must be configured to implement the desired functionality (Heath et al 1998, Williams and Kuekes 2000, Kuekes and Williams 2002, DeHon 2003, Stan et al 2003, Ziegler and Stan 2003, Snider et al 2004, DeHon 2005, Ma et al 2005, Snider 2005, Snider et al 2005, Strukov and Likharev 2005). So we are faced with the challenge of computing with devices that are not only crummy, but dynamical as well. Can reliable Boolean logic systems be built from such crummy devices? Yes, but we argue that at some point as 1 ‘Crummy’ was introduced into the technical lexicon by Moore and Shannon (1954). device dimensions scale down, the overhead and complexity become so costly that performance and density improvements will hit a barrier. In the next section we discuss two frequently proposed strategies for implementing Boolean logic with crummy, dynamical devices—reconfiguration and coding theory—and argue that each has severe scalability problems. This is not suggesting that logic at the nanoscale is not worth pursuing. It clearly is, and semiconductor manufacturers have the economic motivation to continue scaling down as aggressively as their profits permit. Rather we are suggesting that the ‘killer app’ for nanoelectronics lies elsewhere. An alternative computational paradigm—adaptive, recurrent networks—is computationally powerful and requires only two types of components, which we call ‘edges’ and ‘nodes’. Edge to node ratios are typically high, hundreds to thousands, and edges are, unfortunately, difficult to implement efficiently. This difficulty has made these networks extremely unpopular; software implementations are impossibly slow, and hardware implementations require far too much area. 0957-4484/07/365202+13$30.00 1 © 2007 IOP Publishing Ltd Printed in the UK Nanotechnology 18 (2007) 365202 G S Snider In this paper we propose using memristive nanodevices to implement edges, conventional analog and digital electronics to implement nodes, and pairs of bipolar pulses, called ‘spikes’, to implement communication. The tiny size of the nanodevices implementing edges would allow, for the first time, a practical hardware implementation of adaptive, recurrent networks. We suggest that such networks are a better architectural fit to nanoelectronics than Boolean logic circuits. They are robust in the presence of device variations and defects; they degrade gracefully as devices become defective over time, and can even ‘self-heal’ in response to internal change; they can be implemented with simple, crossbar-like structures. Just as importantly, they can be structured to self-organize their computations, sidestepping scalability problems with device characterization, compilation and configuration. Such systems can contain large numbers of defective components, but we will not need to discover where they are—in fact, we will not care where they are. The system can adapt and ‘rewire’ itself around them. 2. Boolean logic is hard with crummy, dynamical devices 2.1. Reconfiguration to the rescue? Several researchers have proposed using a combination of device characterization, defect avoidance, and configuration to handle initial static defects (Stan et al 2003, DeHon 2003, Snider et al 2004, Strukov and Likharev 2005, Snider and Williams 2007). The strategy is a three-pass algorithm: (1) Characterization. Analyze every nanowire and nanodevice in the system and compile a list of resources which are defective (stuck open or stuck closed, shorted, broken, out-of-spec, etc). Such analysis algorithms were used in the Teramac system (Culbertson et al 1997). (2) Defect avoidance. Give the list of defects from pass 1 to a compiler which maps a desired circuit onto the defective fabric, routing around defective components (Culbertson et al 1997, Snider et al 2004, Strukov and Likharev 2005, Snider and Williams 2007). (3) Configuration. Give the mapping determined in pass 2 to a controller that electrically configures each of the mapped components. Since every chip will have a unique set of defects, the above process must be applied to each and every chip. This presents some interesting challenges for manufacturing, since the time required to perform the above steps will contribute to production cost. Characterization (pass 1) is problematic due to device variability—the functional state of a device (or wire) is not necessarily discrete (working versus nonworking) but can lie on a continuum. And characterizing, say, 1012 nanodevices (DeHon et al 2005) in a reasonable amount of time is not likely to be trivial, especially given the bottleneck of pins on the chip. It is not clear if existing characterization algorithms could be applied to systems like this, or how well they would scale. Compilation (pass 2) also presents considerable risk. Compiling circuits onto configurable chips (such as FPGAs) today is a time-consuming process, due to the NP-hard placement and routing problems that lie at the compiler’s core. Even circuits comprising only a few tens of thousands of gates can require several minutes to compile, depending on the degree of optimization needed—and that’s assuming a defect-free target, where a single compilation can be used to manufacture thousands of parts. One proposal for minimizing this problem requires an initial ‘ideal’ compilation onto a hypothetical defect-free fabric, laying out components a little more sparsely than optimal. This would be done only once for each (circuit type, chip type) combination, so one could afford to spend enormous amounts of compute time on this to arrive at this ideal configuration. The configuration of an individual chip on the production line would then be viewed as a ‘perturbation’ of the ideal configuration, with resource allocation shifted as necessary to avoid defects. One might even combine the characterization pass with this pass for further speed improvements. This strategy might be workable. But it is not clear how well this would scale, or how robust this would be in the presence of defect clustering. Configuration (pass 3) is the most straightforward, with the most significant risk being configuration time restrictions due to the pin bottleneck and power dissipation. Note that the above approach of characterize, compile, configure does not handle device death. What happens when a nanodevice stops working and the chip starts computing nonsense? If the nanodevices are reconfigurable, the system can be stopped and reconfigured to work around the newly formed defects. But that assumes additional circuitry to detect the malfunction (e.g. self-checking circuits (Wakerly 1978)), and a companion host processor capable of implementing the three passes in order to reconfigure the chip. Such a processor would have to be fast, reliable (which probably means it would not be implemented with nanodevices), and likely would require a significant amount of memory. Implementing such a coprocessor would seem to negate the benefits that one was presumably getting from the nanoscale circuit in the first place. 2.2. Coding theory to the rescue? Coding theory has been used for decades to robustly transmit information over noisy channels by adding a small amount of redundant information. Can coding theory do the same for logic circuits by adding a small number of redundant gates or components in order to achieve reliable operation? von Neumann (1956), looking forward to nanoscale computation, was perhaps the first to address this question. His approach used a code that replicated each logic gate, and combined the replicated gate outputs in a clever way so that the entire system achieved a desired level of reliability. Although his primary concern was correcting errors induced by transient faults, the approach could also compensate for initially defective devices or ‘device deaths’ as long as the failing devices were randomly distributed and the number did not exceed a threshold (the trade-off being that device deaths would reduce the system’s tolerance of transient faults). The overhead for his scheme could be enormous, though. Replication factors for Boolean logic went to infinity as fault rates approached about 1%. This bound has been improved by later researchers, with replication factors of only 3 claimed for fault rates up to 7.8 × 10−8 for majority gate logic (Roy and